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Art Unit: 2173 

DETAILED ACTION 
Response to Amendment 

This Office Action is responsive to the amendment filed July 23, 2007. Claims 1 , 
4-8, 11-21, 24, and 27 are amended. Claims 3, 10, 23, 26, and 29 are cancelled. 
Claims 30-34 are new. Claims 1, 8, 15, 21, 24, and 27 are independent claims. Claims 
1 , 2, 4-9, 1 1-22, 24, 25, 27, 28 and 30-34 are currently pending. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that 
form the basis for the rejections under this section made in this Office action: 
A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 122(b), by 
another filed in the United States before the invention by the applicant for patent or (2) a patent 
granted on an application for patent by another filed in the United States before the invention by the 
applicant for patent, except that an international application filed under the treaty defined in section 
351(a) shall have the effects for purposes of this subsection of an application filed in the United States 
only if the international application designated the United States and was published under Article 21(2) 
of such treaty in the English language. 

Claims 1,2, 4-9, 11-22, 24, 25, 27, and 28 are rejected under 35 U.S.C. 102(e) 
as being anticipated by Walter et al. (hereinafter Walter), United States Patent 
Application number 2005/0156873. 

Regarding claim 1, Walter teaches a system for mapping captured multimedia 
information onto graphics for insertion into a communication using an Instant Messaging 
(IM) application, wherein the insertion is based on multimedia information (see 
paragraph [0018]; "users can create their own emoticons by adapting many sorts of 
image files to be custom emoticons"), the system comprising an information capture 
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module for capturing the multimedia information in the vicinity of a machine on which 
the user is using the IM application (see paragraph [0028]; "an image selector 308 
captures an image and converts the image to an emoticon"); an information extraction 
and interpretation module communicatively coupled with the information capture 
module, for extracting relevant information from the captured multimedia information 
and interpreting it (see paragraph [0027]; "An exemplary CEE 206 includes a user 
interface 302 that may include a "define custom emoticons" module 304 and a "create 
text message" module 306. The exemplary CEE 206 also includes an image selector 
308, a custom emoticons object store 310, and a transmitter 312, all communicatively 
coupled as illustrated"); and a mapping module communicatively coupled with the 
information extraction and interpretation module, for mapping the interpreted information 
onto a graphic (see paragraph [0023]; "An exemplary CEE 206 also performs real-time 
mapping from the text of a typed message to custom emoticons to be included, 
inserted, substituted, etc., into the message"); an Application Program Interface module 
for the IM application, communicatively coupled to the mapping module, for inserting the 
graphic into the communication in real time using the IM application, said inserting only 
occurring after detecting a trigger from a user (see paragraph [0023]; "In one 
implementation, each node 202, 204 includes an exemplary custom emoticon engine 
"CEE" (e.g., 206, 208). An exemplary CEE 206 allows a user to adopt an arbitrary 
image 210 as a custom emoticon. An exemplary CEE 206 also performs real-time 
mapping from the text of a typed message to custom emoticons to be included, 
inserted, substituted, etc., into the message"; see also paragraph [0033]; "a message 
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transmitter 324 maybe employed to send the character sequence to a destination, i.e., 
within a communication such as an instant message. Each time the character sequence 
associated with a custom emoticon is used in a text message, a custom emoticon 
associated with the character sequence will be substituted for the character sequence 
at the destination, e.g., the receiving client 204", the user sending the instant message 
is the trigger for inserting the graphic). 

Regarding claim 2, Walter teaches that the multimedia information comprises at 
least one of audio information, still image information, and video information. Walter 
teaches that an emoticon can be an arbitrary image (see Abstract, first sentence). 

Regarding claim 4, Walter teaches that the emoticon graphic is predefined by 
the IM application (see paragraph [0030]). 

Regarding claim 5, Walter teaches that the graphic is predefined by a third-party 
application. In Walter, paragraph [0023], it is clear that the invention can be used in 
many different types of applications. 

Regarding claim 6, Walter teaches that the graphic is created by the user (see 
paragraph [0018]). 

Regarding claim 7, Walter teaches that the graphic is created by the user by 
processing captured multimedia information (see paragraph [0024]). 

Claims 8, 9, and 11-14 recite a method with substantially the same limitations as 
the system of claims 1, 2, and 4-7, respectively. Therefore, the claims are rejected 
under the same rationale. It should be noted that claims 11-13 refer to a plurality of 
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emoticons as opposed to a single emoticon, which Walter also anticipates in paragraph 
[0030]. 

Claims 15 and 16 recite a method with substantially the same limitations as the 
system of claims 1 and 3, respectively. Therefore, claims 15 and 16 are rejected under 
the same rationale. 

Regarding claim 17, Walter teaches storing the graphic for use in a later IM 
communication using the application (see paragraph [0030]). 

Regarding claim 18, Walter teaches that the step of processing the received 
captured multimedia information to create a graphic comprises reducing the size of the 
captured multimedia information (see paragraph [0028]). 

Regarding claim 19, Walter teaches that the step of processing the received 
captured multimedia information to create a graphic comprises reducing the resolution 
of the captured multimedia information (see paragraph [0028]). The resolution of the 
emoticon after reduction in size is inherently reduced. 

Regarding claim 20, Walter teaches that the step of processing the received 
captured multimedia information to create a graphic comprises selecting a frame from a 
plurality of frames of the captured multimedia information (see paragraph [0049]). 

Claims 21 and 22 recite a system with substantially the same limitations as the 
system of claims 1 and 2, respectively. Therefore, the claims are rejected under the 
same rationale. 
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Claims 24 and 25 recite a method with substantially the same limitations as the 
system of claims 1 and 2, respectively. Therefore, the claims are rejected under the 
same rationale. 

Claims 27 and 28 recite a system with substantially the same limitations as the 
system of claims 1 and 2, respectively. Therefore, the claims are rejected under the 
same rationale. 

Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set. 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the • 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

Claims 30-34 are rejected under 35 U.S.C. 103(a) as being unpatentable over 

Walter (2005/0156873) supra and Day et al. (hereinafter Day), United States Patent 

number 7,039,676. 

Regarding claim 30, Walter teaches every limitation of claim 30 except that said 
graphic represents motion by said user. However, Day teaches a system that analyzes 
user motion to determine graphics to insert into real-time instant messages (see Day 
column 3 line 36 through column 4 line 2; "The system, method and program of the 
invention automatically generates input into chat room software that represents an 
actual physical gesture made by a participant in a real time communication over a 
network, such as a "live" chat session or an instant messaging communication"). It 
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would have been obvious to one of ordinary skill in the art at the time the invention was 
made to use the motion and gesture analysis system of generating emoticons of Day to 
generate the custom icons in the invention of Walter for the purpose of making it easier 
for users to generate custom emoticon graphics for transmitting over instant message 
sessions. 

Regarding claim 31, Day teaches that said trigger is a gesture by said user (see 
Day column 3 line 36 through column 4 line 2; "A video camera, utilized in connection 
with the participants' computer system, captures the real time gestures made by the 
participant, such as a wave, a shoulder shrug, a nodding of the head, and inputs the 
captured video images into the computer system of the participant', receiving a gesture 
is the action that triggers the system to generate an emoticon graphic). 

Regarding claim 32, Day teaches that said relevant information extracted by 
said information extraction and interpretation module is in a non graphic format (see 
Day column 8 lines 16-26; "since the configuration process has defined each gesture 
with a corresponding action, a database or table 300 (FIG. 3) becomes populated with 
the gesturing events 301, state of gesture 302, and corresponding action 303 and 
parameter of the action 304, i.e., the content to be transmitted for the gesturing event'). 

Regarding claim 33, Day teaches that said relevant information extracted by 
said information extraction and interpretation module is mapped to one of a preselected 
group of graphics, including graphics representing a smile, a frown and a wink (see Day 
column 7 lines 48-58; "the automatic gesture software may provide a set of available 
gestures, e.g., wave hand, smile, frown, wink, shrug, nod, for which the user may 
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designate the action (announce, insert text, insert graphic,) and the parameter of the 
action (e.g., the content or translation of the gesture)"). 

Regarding claim 34, Day teaches that said relevant information extracted by 
said information extraction and interpretation module is an article worn by said user (see 
Day column 6 lines 15-32; "The imaging software analyzes various features of a 
participant from captured video frames generated by video camera 115. For example, 
the imaging software may discern any one or more of the following features including, 
but not limited to, the head, eyes, mouth (lips), shoulders, arms, and hands. For 
example, the imaging software can detect whether the head nods up and down in 
successive frames, or if there is a prolonged "wink" in one eye, or if the mouth makes a 
smile or frown, or if the shoulders "shrug", or if an arm or hand moves across the 
captured video frames such as in depicting a wave or other gesture", the imaging 
software described by Day is capable of detecting and interpreting articles worn by a 
user). 

Response to Arguments 

Applicant asserts that Walter does not teach graphics being inserted into a 
communication "in real time". The examiner respectfully disagrees. 

In Walter paragraph [0023], Walter teaches "An exemplary CEE 206 also 
performs real-time mapping from the text of a typed message to custom emoticons to 
be included, inserted, substituted, etc., into the message". 
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Applicant further asserts that Walter does not teach a "trigger" which initiates 
insertion of the graphic into the message. The examiner respectfully disagrees. 

In Walter paragraph [0033], Walter teaches "Each time the character sequence 
associated with a custom emoticon is used in a text message, a custom emoticon 
associated with the character sequence will be substituted for the character sequence 
at the destination, e.g., the receiving client 204". The user sending the instant message 
is the trigger for inserting the graphic into the message. 

The new claims introduce subject matter which the examiner believes would 
have been obvious in light of gesture-aware instant messaging applications that existed 
at the time the invention was made, such as the software taught by Day. 

Conclusion 

THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
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the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Stephen Alvesteffer whose telephone number is (571) 
270-1295. The examiner can normally be reached on Monday-Friday 9:30AM-6:00PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, John Cabeca can be reached on (571)272-4048. The fax phone number for 
the organization where this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Stephen Alvesteffer , — / - 
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